140 


An Examination of the Relationship Between Online Class Size and Instructor 

Performance 


Chris Sorensen, Ashford University, San Diego, California, USA 

Abstract 

With no physical walls, the online classroom has the potential to house a large number of 
students. A concern by some is what happens to the quality of instruction in courses with high 
enrollments. The purpose of this research was to examine online class size and its relationship 
to, and potential influence on, an instructor’s performance. Results were mixed indicating that 
class size had a positive relationship with some the variables meant to measure online instructor 
performance and a negative relationship with others. Online class size was seen as having the 
most concerning relationship and potential influence on an instructor’s ability to provide quality 
feedback to students and for his/her expertise to be used consistently and effectively. 

Keywords: Online class size, Instructor performance, quality instruction, online learning, higher 
education 

INTRODUCTION 

Class size has been a topic of debate for some time now. If instructors were asked about 
their preferences in class size, it probably would not be a surprise to many to find that instructors 
prefer smaller classes. The rationale being that when there is a smaller student-to-instructor ratio, 
the instructor has the opportunity to provide more individualized instruction and interaction. This 
in turn, could result in a more positive and higher quality learning experience for the student. In 
addition, smaller classes may be seen as easier to manage. As online education has become more 
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popular, it appeared that a major advantage of this form of learning is that a larger number of 
students can be taught in a single class. With online education, there are no physical limitations 
to the number of students who can be put into “a classroom,” as the virtual space is limitless. 
This scenario has resulted in some educators scrutinizing class sizes of online courses. In 
general, research examining online class size has been mixed as far as what constitutes the ideal 
class size that is conducive to teaching and learning. 

The purpose of this study was to examine online class size and its relationship to and 
potential influence on an instructor’s performance, which could be an indication of quality of 
instruction. Instructor performance was measured by the results from online course peer reviews 
of faculty in the areas of: fostering critical thinking, providing instructive feedback, maintaining 
high expectations, establishing relationships, and the use of instructor expertise. Class size was 
defined as the number of students in the course once the course had ended. 

LITERATURE REVIEW 

Taft et al (2011) present a summary of several studies examining the recommendations 
for online class sizes. Results of past studies appear to be mixed. Some studies suggested 
establishing smaller class sizes that ranged from 4 tol2 (Berry, 2008; Blood-Siegfried et al., 
2008; Buckingham, 2003; Rovai, 2002; Schellens and Valcke, 2006). Qiu et al. (2012) found 13 
to 15 students to be the optimal class size when examining note reading and writing loads in 
online graduate-level courses. Some studies suggested instituting slightly larger class sizes that 
ranged from 20 to30 (Arbaugh and Benbunan-Fich, 2005; Arbaugh, 2005; Dykman and Davis, 
2008; Keeton, 2004). Weick (2014) presents one university’s strategy, which is to keep their 
online courses similar in size to their “in-seat” courses (approximately 20 students). The 
university believes that a student-to-faculty ratio around 20 to 1 allows for better faculty 
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interaction with students. Other studies offered no recommendations (Andersen and Avery, 

2008; DiBiase and Rademacher, 2005). 

Previous studies have been conducted, which examined online class size in relation to 
various factors in the online classroom. Orellana (2006) presents a study investigating class size 
in regards to interaction. Upon surveying 131 instructors, it was found that, on average, a class 
size of 15.9 was seen by the instructors as optimal for achieving the highest level of interaction. 
The average class size in this study was 22.8. Ravenna (2012) proposes that larger class sizes can 
have a negative effect on instructor-student interaction. It is suggested that in courses with large 
student-to-instructor ratios, smaller groups should be created with a facilitator or group 
moderator. This will help to simulate a smaller student-to-instructor ratio. As part of a doctoral 
thesis, Reonieri (2006) examined issues surrounding effective online discussions. When the 
students and faculty were surveyed on their perception in taking part in online discussions in a 
class that was too small, 60% of students and 78% of faculty noted that there was a negative 
effect on the discussion (i.e. there were a limited number of perspectives or too few interactions). 
When the students and faculty were surveyed on their perception of taking part in online 
discussions in a class that was too large, 94% of students and 94% of faculty noted that there was 
a negative effect on the discussions (i.e. being overwhelmed by the number of responses or 
having difficulty following the discussions). This study defined a small online class as having 5- 
10 students and a large online class has having 15 or more. Harmon et al. (2014) state that it can 
be challenging to have meaningful discussions as online class size increases. 

Another potential factor affected by class size is the amount of time required by 
instructors to devote to an online course. It has been suggested that online courses require more 
time (Bair, 2011; Berge & Muilenburg, 2001; Cho & Berge, 2002; Meyer, 2012; Mupinga & 
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Maughan, 2008; Sword, 2012). Bender, Wood, and Vredevoogd (2004) compared time-on-task 
for instructors teaching a traditional face-to-face course verses an online version of the same 
course. There were 111 students enrolled in the traditional course (facilitated by 38 
undergraduate teaching assistants) and 18 in the online course (facilitated by five teaching 
assistants). Results revealed more total time was spent in the traditional course due to the higher 
number of students; however, when the results were analyzed on a per student basis, time-on- 
task was higher in the online course. Mupinga and Maughan (2008) present an exploratory study 
examining common practices on the treatment of workload issues at higher education institutions 
with a particular focus on online instructors. Among the workload issues was class size. It was 
found that class size varies depending on the institution. Many institutions had similar online 
class sizes as compared to their face-to-face counterparts while others had a cap on class size 
(most ranging from 20 to 50). Despite the class size, it was found that faculty would spend more 
hours attending to online courses than face-to-face courses. 

Providing quality instruction will often take more time. For most instructors in higher 
education, teaching is just a component of what they do on a daily basis. It is also believed by 
some instructors that the quality of online instruction is questionable for large class sizes (Parker, 
2003; University of Illinois, 1999 as cited in Orellana, 2006). If teaching an online course does 
require more time and instructors are teaching an online course with a higher number of students, 
instructors might be tempted to “water down” the quality of instruction so that they can complete 
their teaching duties (i.e. grading assignments) in a timely manner. 

Research Questions 

The following served as research questions for this study: 

I. What correlations exist between class size and an instructor’s overall peer review score? 
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II. What correlations exist between class size and: 

a. Fostering Critical Thinking 

b. Instructive Feedback 

c. High Expectations 

d. Establishing Relationships 

e. Instructor Expertise 

III. To what degree might online class size predict performance in: 

a. Fostering Critical Thinking 

b. Instructive Feedback 

c. High Expectations 

d. Establishing Relationships 

e. Instructor Expertise 

f. The average peer review score 
METHODS 

This study was non-experimental in nature and utilized data that was collected during the 
College of Education (COE) 2013 Peer Reviews of part-time online instructors at a large for- 
profit University. Prior to carrying out the study, it underwent a review by the Institutional 
Review Board of the researcher’s affiliated university. The methods were deemed appropriate 
given the study’s purpose and research questions and there were no ethical concerns identified. 

The sample size consisted of 380 part-time online instructors. Data was collected via 
classroom walk-through observations of the online instructor’s courses. The classroom walk¬ 
through observations would be equivalent to more traditional qualitative data collection methods 
centered on participant observation. To facilitate the classroom walk-through observations, a 
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peer review rubric was used. The rubric, developed by a committee consisting of the university’s 
administrators and faculty, was specifically created for evaluating online instructor performance 
at the for-profit university. The rubric criteria and definitions emerged as a result of research that 
was conducted around the best practices in online teaching. Five criterion were developed that 
the committee felt addressed key areas reflecting online teaching (Fostering Critical Thinking, 
Instructive Feedback, High Expectations, Establishing Relationships, and Instructor Expertise) 
along with five levels of performances (Not Observed, Beginning, Developing, Proficient, 
Distinguished). Each level of performance had a description of what an instructor’s performance 
would look like at that level for each criterion. Table 1 presents the general rubric structure. 
Table 1 


General Structure of the Peer Review Rubric 


Criteria 

Not Observed 

Beginning 

Developing 

Proficient 

Distinguished 

Fostering 

Critical 

Thinking 

Description of 
performance 

Description 

of 

performance 

Description 

of 

performance 

Description 

of 

performance 

Description of 
performance 

Instructive 

Feedback 

Description of 
performance 

Description 

of 

performance 

Description 

of 

performance 

Description 

of 

performance 

Description of 
performance 

High 

Expectations 

Description of 
performance 

Description 

of 

performance 

Description 

of 

performance 

Description 

of 

performance 

Description of 
performance 

Establishing 

Relationships 

Description of 
performance 

Description 

of 

performance 

Description 

of 

performance 

Description 

of 

performance 

Description of 
performance 

Instructor 

Expertise 

Description of 
performance 

Description 

of 

performance 

Description 

of 

performance 

Description 

of 

performance 

Description of 
performance 


To provide an idea as to what the performance descriptions look like, Table 2 presents the 
descriptions for each of the criteria at the Distinguished level on the peer review rubric 


(Wardlow et al., 2011): 
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Table 2 


Distinguished Descriptions for the Peer Review Criteria 


Criteria 

Distinguished Descriptions 

Foster Critical 

Challenging students to elaborate on their thoughts, question their 

Thinking 

assumptions, examine biases, communicate in a clear and concise manner, and 
defend their positions throughout the course. 

Instructive 

Feedback 

Providing feedback that challenges and inspires students, while providing 
specific suggestions to improve the quality of their work and thinking. 

High 

Expectations 

Demonstrating high expectations throughout the course, while holding students 
accountable for insightful exchanges and high quality performance on 
assignments, and promoting active engagement in their own learning. 

Establishing 

Relationships 

Creatively uses available tools (Announcements, Instructor Guidance, Faculty 
Expectations, Ask Your Instructor, Emails, Discussion Forum) and strategies 
to enhance relationships, creating a community of learners willing to take risks 
and actively engage with one another. 

Instructor 

Expertise 

Effectively and consistently utilizes expertise in subject matter by providing 
personal experiences, connecting course knowledge to real-world examples. 
Enhances course content and resources to encourage student comprehension 
and application of course learning outcomes. 


Of the 380 courses that were reviewed, 217 were undergraduate courses and 163 were 
graduate courses. All courses were taught within the COE. The following presents an overview 
of the peer review process and how numerical peer review scores were determined based on the 
classroom walk-through observations: 

Step 1: Each part-time instructor being reviewed was assigned to two full-time faculty 

members. Twenty-four full-time faculty members were involved in the 2013 COE 
Peer Reviews. Peer reviews occurred on a monthly basis throughout 2013 and 
each full-time faculty member would typically conduct four to six peer reviews a 
month. Occasionally a faculty member might have been asked to conduct as many 
as eight or as few as one. Peer reviewers participated in at least one training 
workshop and all full-time faculty were experienced online instructors. In 
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addition, resources such as check lists, exemplars, and rubrics were made 
available to the faculty. 

Step 2: Each full-time faculty member individually reviewed the part-time instructor’s 
course using the COE Peer Review Rubric. On average, the review of a single 
course took about an hour. For each criterion, a level of performance was noted. 
Levels of performance where then translated into a numerical score (0= Not 
Observed, 1= Beginning, 2= Developing, 3= Proficient, 4= Distinguished) for 
each criterion. The five numerical scores were then averaged together to create a 
single overall peer review score. 

Step 3: After reviewing the course, the two full-time faculty members scheduled and 
conducted a phone call, usually 30 to 40 minutes, to discuss the part-time 
instructor’s performance and determine a single agreed upon score for each of the 
five criteria on the peer review rubric. No averages were calculated for the 
individual criteria. For example, if one full-time faculty member scored foster 
critical thinking at a 2 and the other full-time faculty member scored foster critical 
thinking at a 3, they would have to come to a consensus on either 2 or 3 (not 2.5). 
Once a single a score was agreed upon and assigned to each of the five criteria, 
the five scores were averaged together to create a single overall peer review score. 
In the end, each part-time instructor had six scores, one for each of the rubric 
criteria and one overall average score. 

Results 

The program PASW Statistics 18 was used in the data analysis. The following statistical 
analyses were used to answer the research questions: Descriptive Statistics, Pearson Correlation 
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Coefficients, and Linear Regression. For the purpose of data analysis, the following 
abbreviations were used for each of the variables: CS (class size), CT (fostering critical thinking), 
IF (instructive feedback), HE (high expectations), ER (establishing relationships), IE (instructor 
expertise), and Avg (overall average peer review score). In assessing the reliability of the five 
rubric peer review scores (CT, IF, HE, ER, IE) a Cronbach’s Alpha of .87 was calculated. This 
would suggest high reliability in the scores that were determined by the full-time faculty. 

Tables 3 and 4 present the descriptive statistics and correlations for the variables being 
examined. 

Table 3 

Descriptive Statistics for Class Size, Fostering Critical Thinking, Instructive Feedback, High 
Expectations, Establishing Relationships, Instructor Expertise, and Average Peer Review Score 


Variables 

n 

Min 

Max 

M (SD) 

CS 

380 

1 

29 

15.88 (7.02) 

CT 

380 

1 

4 

2.52 (.75) 

IF 

380 

0 

4 

2.48 (.80) 

HE 

380 

1 

4 

2.68 (.76) 

ER 

380 

0 

4 

2.75 (.77) 

IE 

380 

1 

4 

2.74 (.77) 

Avg 

380 

.8 

4 

2.63 (.63) 


Table 4 

Correlations between Class Size, Fostering Critical Thinking, Instructive Feedback, High 


Expectations, Establishing Relationships, Instructor Expertise, and Average Peer Review Score 


Variables 

1 

2 

3 

4 

5 

6 7 

1. CS 

— 






2. CT 

.008 

— 





3. IF 

-.055 

.599** 

— 




4. HE 

.008 

.622** 

.697** 

— 



5. ER 

.024 

.563** 

.523** 

.624** 

— 


6. IE 

-.081 

.527** 

.510** 

.573** 

.583** 

— 

7. Avg 

-.024 

.809** 

.820** 

.862** 

.806** 

.782** 


a Note. N = 380 

b ** Correlation is significant at the 0.01 level 
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A series of linear regression models were conducted to estimate the predictive affects of 
CS on CT, IF, HE, ER, IE, and Avg. Based on the results of the linear regressions, equations 
were constructed that could be used to predict values for each of the variables based on CS. 
Tables 5, 6, and 7 summarize these results. 

Table 5 


2 

A Summary of the Variance (R ) that is Accounted for by CS in each of the Dependent Variables 


Dependent Variables 

R 

R" 

Adjusted R“ 

Std. Error of the 
Estimate 

CT 

.008 

.000 

-.003 

.747 

IF 

.055 

.003 

.000 

.803 

HE 

.008 

.000 

-.003 

.764 

ER 

.024 

.001 

-.002 

.768 

IE 

.081 

.007 

.004 

.769 

Avg 

.024 

.001 

-.002 

.629 


a Note. N = 380 


Table 6 

A Summary of the Linear Regression Models with Class Size Predicting Fostering Critical 
Thinking, Instructive Feedback, High Expectations, Establishing Relationships, Instructor 
Expertise, and Average Peer Review Score 




Unstandardized 

Standardized 



Dependent 


Coefficients 

Coefficients 



Variable 


B 

Std. Error 

Beta 

t 

Sig- 

CT 

(constant) 

2.05 

.095 


26.393 

.000 

CS 

.001 

.005 

.008 

.155 

.877 

IF 

(constant) 

2.577 

.102 


25.237 

.000 

CS 

-.006 

.006 

-.055 

-1.075 

.283 

HE 

(constant) 

2.665 

.097 


27.443 

.000 

CS 

.001 

.006 

.008 

.152 

.879 

ER 

(constant) 

2.708 

.098 


27.741 

.000 

CS 

.003 

.006 

.024 

.467 

.641 

IE 

(constant) 

2.883 

.097 


29.519 

.000 

CS 

-.009 

.006 

-.081 

-1.575 

.116 

Avg 

(constant) 

CS 

2.668 

-.002 

.080 

.005 

-.024 

33.377 

-.472 

.000 

.637 
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Table 7 

Equations Constructed Using Class Size to Predict Fostering Critical Thinking, Instructive 
Feedback, High Expectations, Establishing Relationships, Instructor Expertise, and Average 


Peer Review Score 


Dependent 

Variable 

Equation 

CT 

Y = 2.505 + (.001 )A, where Y is the predicted CT score and A is the CS 

IF 

Y = 2.577 + (-.006)A, where Y is the predicted IF score and X is the CS 

HE 

Y = 2.665 + (.001)A, where Y is the predicted HE score and A is the CS 

ER 

Y = 2.708 + (.003)A, where Y is the predicted HR score and A is the CS 

IE 

Y = 2.883 + (-.009)A, where Y is the predicted IE score and A is the CS 

AVG 

Y = 2.668 + (-.002)A, where Y is the predicted Avg score and A is the CS 


Discussion 

Based on Table 3, it is seen that the average class size was 15.88 students with the 
smallest class consisting of one student and the largest class consisting of 29 students. When 
examining the five variables meant to measure instructor performance, the lowest average was 
associated with instructor feedback (approximately 2.48) and the highest average was seen with 
establishing relationships (approximately 2.75). The overall average peer review score for the 
380 part-time online faculty members that were reviewed was 2.63, with the lowest individual 
average being a 0.8 and the highest individual average being 4.0. All of the averages scores 
presented in Table 3 would indicate that the average instructor performance in each area was 
between developing and proficient on the rubric used to evaluate them. 

When addressing research question one, “What correlations exist between class size and 
instructor’s overall peer review score?,” analysis revealed the correlation to be -.024 (see Table 
4). This value was found to be statically non-significant. What is important to note is that the 
correlation is negative. This suggests that, as class size increases, the instructor’s overall peer 
review score representing the instructor’s overall performance and an indication of the quality of 
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instruction, decreases. The negative relationship seen here is in par with previous research which 
revealed negative results such as increased class sizes having a negative effect on discussion 
interactions (Reonieri, 2006) or the increased time needed to devote to an online course (Berge 
& Muilenburg, 2001; Cho & Berge, 2002; Mupinga & Maughan, 2008) which could be further 
magnified by a larger class size. 

Research question two asks, “What correlations exist between class size and: Fostering 
Critical Thinking, Instructive Feedback, High Expectations, Establishing Relationships, and 
Instructor Expertise?” When viewing Table 4, class size was seen to have the following 
correlations with the five variables being examined: .008 (CT), -.055 (IF), .008 (HE), .024 (ER), 
and -.081 (IE). Like the correlation between class size and the average peer review scores, these 
five correlations were also found to be statistically non-significant. What may be of interest are 
the negative correlations that were observed between particular variables and class size, namely 
instructor feedback and instructor expertise. Again, although not statistically significant, the 
negative correlations suggest that as class size increases, the instructor’s performance in the areas 
of instructor feedback and instructor expertise decreases. These results provide support for 
previous research that questioned the quality of online instruction in classes with larger class 
sizes (Parker, 2003; University of Illinois, 1999 as cited in Orellana, 2006; Taft et al., 2011) 

Although not directly related to a research question, it is worth mentioning that all 
correlations between the five rubric-based variables (CT, IF, HE, ER, IE) were positive (as one 
increases the other increases) and all correlations were found to be statistically significant at 
the .01 level. This suggests that there is a strong relationship between the five variables, with the 
strongest correlation existing between instructor feedback and high expectation (.697). One 
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might conclude that online instructors who hold higher expectations for their students also 
provide higher quality feedback. 

At the University where this study was conducted, there are caps placed on class sizes for 
both undergraduate and graduate courses with the upper limit being 30 students. As a result, this 
raises the question, “What would the correlations look like with either an increased upper limit or 
no limit?” Would the same results be seen in courses that had 40, 50, or 60 students? With no 
physical limitations, it could be very tempting to enroll as many students as possible into one 
online course. 

Research question 3, “To what degree might online class size predict: Fostering Critical 
Thinking, Instructive Feedback, High Expectations, Establishing Relationships, and Instructor 
Expertise, and the overall Average Peer review score?,” addresses the predictability of CT, IF, 
HE, ER, IE, and Avg based on CS. When the R" values in Table 5 are examined, it is seen that 
very little variability in any of the peer review scores is accounted for by class size. Class size 
accounted for approximately 0.3% of the variability in IF, 0.1% of the variability in ER, 0.7% of 
the variability in IE, and 0.1% of the variability in Avg. As far as CT and HE, class size 
accounted for 0% of the variability. Based on this, it can be suggested there may be other 
variables that have a greater influence on the variability of the CT, IF, HE, ER, IE, and Avg peer 
review scores. If these results are interpreted as is within the context of this study, it can be said 
that variability due to class size was seen the most in IE, followed by IF, then ER and Avg, and 
finally no variability in CT and HE. 

Table 6 presents the results of the linear regression models and the predicted change in 
the dependent variables by CS. For every one unit change in CS (i.e. one student), the following 
changes are predicted in each of the scores for the dependent variables: .001 (CT), -.006 
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(IF), .001 (HE). .003 (ER), -.009 (IE), and -.002 (Avg). Six equations (See Table 7) were 
constructed from the linear regression results. Table 8 presents hypothetical predictions using the 
six equations based on a class containing 10 students verses 50 students to help illustrate the 
application of the equations. For example, if an instructor had an online course with 10 students, 
the instructor’s CT score would be predicted to be 2.52. If the same instructor taught the same 
online course but had 50 students, his/her CT score would be predicted to be 2.55. Based on the 
rubric that was used in this study during the peer review process, both of these scores would 
indicate that the instructor was performing between Developing (2) and Proficient (3) in the area 
of CT. 

Table 8 


Examples of Applying the Linear Regression Equations 


Outcome Variables 

Hypothetical Class Size 
and Predicted Scores 

10 Students 50 Students 

The Difference in 
— Scores 

CT 

2.52 

2.55 

0.03 

IF 

2.52 

2.28 

-0.24 

HE 

2.68 

2.72 

0.04 

ER 

2.74 

2.86 

0.12 

IE 

2.79 

2.43 

-0.36 

Avg 

2.65 

2.57 

-0.08 


As presented in Table 8, the only areas where there is a decrease in scores and an increase 
in CS are with IF, IE, and Avg. These predictions coincide with the correlations between these 
variables and class size, which were all found to be negative. Furthermore, when we look at the 
magnitude of the changes and the individual equations (see Tables 6 and 7), IF and IE had the 
two largest changes for each one unit change in CS (-.006 and -.009 respectively). This is also 


illustrated in Table 8 where the largest difference in scores between a class with 10 students and 
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a class with 50 is seen in IF and IE. These results may indicate that CS has a greater influence 
on IF and IE as compared to the other three rubric variables meant to measure performance. 

Limitations 

There are a few potential limitations to this study. First, data was collected from part-time 
faculty. These instructors could be employed elsewhere and thus may not have placed teaching 
their online course at this particular university as a high priority. This in turn could be reflected 
in the instructor’s peer review scores. Second, data was collected from a large for-profit 
University and only included courses within the COE. As a result, it might be difficult to 
generalize the results to any institution that offers online courses. Third, the value for class size 
reflected the number of students present at the end of the course. There may have been students 
who were actively participating early on but, for some reason, dropped the course. In addition, 
there may have been students who missed the drop date and stayed in the course but did not 
participate (i.e. stopped completing assignments but remained on the roster). Both of these 
situations could have created inconsistent instructor performance. Fourth, there was a class size 
limit placed on the courses that were reviewed as part of the peer review. This may affect the 
ability to apply the predications to online courses that exceed 30 students. 

CONCLUSION 

This study, like previous ones, revealed mixed results with online class sizes having a 
positive relationship with some variables and a negative relationship with others. As with any 
correlation, we cannot conclude a causal relationship between class size and any of the five 
variables that were examined from the overall peer review scores in this study. Based on the non¬ 
significant correlations, we can conclude that there is not enough evidence in this particular 
sample to show a statistical significant correlation between class size, the five rubric-based 
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variables, or the overall peer review score meant to represent instructor performance and quality 
of teaching. 

What may provide practical significance and be of interest to individuals working to 
improve the quality of online learning are the results that indicate negative relationships. It was 
found that changes in CS predicted the greatest change in IE and IF (both being negative) when 
compared to CT, HE, and ER, which were all seen as positive. Furthermore, CS accounted for 
more variability in the individual IE and IF scores when compared to CT, HE, and ER scores. 
This study revealed that as the class size of an online class increases, an instructor’s expertise 
may not be used as effectively or consistently, the quality of feedback that the instructor provides 
to students may decrease, and the instructor’s overall performance is seen as decreasing. This 
could raise concern for some, as feedback and expertise are two critical areas that support student 
learning. It is through feedback and the sharing of expertise that an online instructor can clear up 
misunderstandings in course content, guide students in acquiring new knowledge, and address 
those “teachable moments” that are often referred to in classroom teaching. 

Recommendations and Future Studies 

There are several recommendations for future studies examining online class size and 
instructor performance based on the results of this study. First, conduct similar studies with 
online class size that have caps that exceed 30 students. The courses that were part of the peer 
review in this study had a class size limit of 30 students and as a result, the equations constructed 
to predict instructor performance may have been influences by this limitation. Although table 8 
presented an example of the predicted peer review scores in a course with 50 students, research 
should be conducted with class sizes over 30 students to confirm or dispute the results of this 
study. Second, conduct similar studies that include full-time faculty, universities that are not-for 
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profit, and/or courses that are taught in other colleges outside of the COE. This study included 
part-time instructors at a for-profit university teaching online courses within the COE. Based on 
this, the generalizability of the results could be limited. Third explore the relationships that were 
seen as positive. For example, as CS increases, what may account for the increase in an 
instructor’s peer review scores in the areas of CT, HE, and ER? One might have expected to see 
a similar negative relationship that was seen with IF, IE, and Avg. 
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